Chapter 1.1. Introduction

1.1.1. - Game Narrative and Musical Development

Many video games depend on some form of narrative development to create an engaging experience (Sinclair, 2014; Wolf, 2005). This thesis addresses game music’s relationship with video game narrative, so the term ‘narrative’ should be defined within the medium of video games. Salen and Zimmerman quote a general definition of narrative provided by literary theorist, J. Hillis Miller:

“There must be, first of all, an initial situation, a sequence leading to a change or reversal of that situation, and a revelation made possible by the reversal of the situation. Second, there must be some use of personification whereby character is created out of signs— for example, the words on the page in a written narrative, the modulated sounds in the air in an oral narrative. However important plot may be, without personification there can be no story-telling….Third, there must be some patterning or repetition of key elements” (Miller, 1990, from Salen & Zimmerman, 2004, 371).

Salen and Zimmerman expand on Miller’s definition to fit the context of narrative in video games:

“Miller's definition is in some ways a formal approach to narrative. Events, characters, and patterned action describe the qualities of the narrative object, rather than the experience of that object… [O]ur intention is not just to arrive at a formal understanding of narrative (What are the elements of a story?) but instead an experiential one (How do the elements of a story engender a meaningful experience?)… [O]ur concern is with the experience of players: their internal state of mind, and the relationships they form with each other and with the dynamic system of a game. If we shift Miller's definition of narrative into an experiential framework, we can begin to discuss narrative play [as the experience of various] components of games— that they are complex sensual and psychological systems, that they create meaning through choice-making and metacommunication, that they sculpt and manipulate desire—[as] tools for crafting narrative experiences” (Salen & Zimmerman, 2004, 375).

As I periodically reference game music’s relationship with narrative, I am referring to Salen and Zimmerman’s concept of how various game elements are altered or developed by player actions, resulting in a player-driven experience. The pacing and development of narrative throughout a video game is reliant on the timing of player actions. Narrative-driven games often allow the player to explore in their own time, progressing down their chosen sequence of events. In many (but not all) narrative-driven games, adaptive music is used to develop musical material congruently with the player’s progression to support the current game-state.

The player’s unpredictability presents a challenge for congruently communicating the game’s narrative with music, as narrative-driven music traditionally relies on a pre-planned temporal development of musical elements (generalized from: Maus, 1987; Kramer, 1991; Almén, 2008, 3-10). That is not to say that all games or game music must be narratively driven. However, I am specifically looking at examples of adaptive game music that do attempt to support indeterminate narrative-driven gameplay. In Chapter 4, through three case studies, I analyze the extent to which three different games utilize dynamic music to support the player’s narrative-driven gameplay experience.

This thesis aims to offer a possible solution to the conflict between music’s traditionally linear compositional devices that are used for communicating musical narrative (Almén, 2008), and narrative-driven video games’ indeterminate, non-linear, interactive qualities. This is presented as a new approach for designing dynamic music systems.

Composers have confronted this challenge in a number of different ways. The current practice is to render compositions into segmented audio files. The segmented audio files are implemented into the game and triggered to play in response to player actions or changing game states. Current practices do not solve the conflict that I have described above. Current dynamic music techniques often operate through relatively simple if, then, until, logic systems (Collins, 2009, 5). This type of system often restricts how music is able to develop structurally, resulting in the conflict between structure and interactivity, as is shown throughout each chapter of the thesis.

If the player enters a cave,
then loop the ‘scary music segment’,
until the player exits the cave.
If the player exits the cave,
then play the ‘scary music stop segment’.

This type of logic system could result in the ‘scary cave music’ getting stuck and looping indefinitely, or potentially containing a disjunct transition to the next music segment (Medina-Gray, 2014, 2). There are potentially negative effects when music becomes stuck in repetitious loops during gameplay. Composers often seek to ease these negative effects by reducing the amount of musical repetition in games (Berndt, Dachselt, Groh, 2012). Medina-Gray states that a player’s reaction to music’s repetition or transitional disjunction may also be dependent on other factors. “…[P]layers’ expectations of and reactions to game music rely at least partly on their previous experiences with the particular game in question, as well as various inter-textual factors such as genre, time period, and technology” (Medina-Gray, 2019).

I believe that the discussion of musical disjunction and repetition in games is closely tied to the challenges involved with making adaptive music sound smooth, seamless, and non-repetitive. That is not to say that music must always try to achieve these qualities. Sometimes musical disjunction is a mechanism used in various types of games to communicate certain game-state changes to the player (Phillips, 2014, 53). But in the context of narrative gameplay where music attempts to develop alongside the player’s progression, disjunction can be harmful to the player’s experience (Summers 2016b, 79–80, from Medina-Gray, 2019), as can over-repetition.

As Medina-Gray has noted, the degree at which listeners perceive and react to musical development is partly dependent on expectations of musical genre (Medina-Gray, 2019). An abrupt muted gong passage emerging from a soft, sparse, non-metric bed of micro-tonal percussion improvisations, which transitions into a more metrically structured section of percussion music, might not be described as a disjunct musical transition. However, an abrupt muted gong passage suddenly interrupting a choral piece in the style of Bach, which transitions into a micro-tonal section of percussion music, could be described as a disjunct musical transition.

When adaptive video game music abruptly changes, and the change is aesthetically inappropriate to the conventional syntax of the initial musical style, then the musical transition could be described as disjunct. When adaptive music suddenly changes, but does so in a way that adheres to the conventional syntax from the initial musical style, then the musical transition could be described as smooth.

While there are a variety of existing approaches, with some composers seeking to ease the amount of repetition and increase smoothness through more complex logic systems or clever compositional approaches, the conflict created by the music’s narrative development being driven by unpredictable player interactivity is still prevalent in contemporary games. This will be identified by observing and analyzing various games throughout the thesis.

My research uses current industry-standard practices as a starting point for developing my own DMS (dynamic music system) design. Chapter 2 reviews the history of dynamic game music to identify why current practices have become standard. Chapter 3 reviews various dynamic music techniques and how they are used in other games. Chapter 3 also includes four playable games that I have written music for to allow the reader to experience how the various dynamic music techniques function. Chapter 4 reviews existing methods for analyzing DMSs in games and presents my own analytical methodology. My analytical method is used to conduct three case studies for gameplay scenarios in “Tomb Raider: Definitive Edition” (Crystal Dynamics, 2013), “Dark Souls 3” (FromSoftware, 2016), and “Halo: Combat Evolved” (Bungie, 2001). The culmination of my research and analysis is a DMS prototype in Chapter 5. The DMS prototype is a short playable game that I have written over thirty minutes of music for. The music has been organized into two-hundred twelve individual segments and implemented into the game so that the music’s development adapts to player actions. The playable games within the thesis were made in collaboration with game developer Phi Dinh, who contributed to the art and programming of the games. Phi did not contribute to the creation of any audio assets or audio implementation.



1.1.2. - Research Question

Narrative-driven games often attempt to create meaningful experiences, which are dictated by the player’s actions. Within the player's experiences, must the music sacrifice its potential to communicate narrative development for the sake of being adaptive? Must the output of an adaptive musical structure result in music that is constantly interrupted by the repetition of its smaller parts? One way of thinking about these rhetorical questions might be to imagine extracting the music from a short segment of recorded gameplay and asking whether or not the musical narrative represents the player’s experienced gameplay narrative, or if the music sounds like a sequence of loops that happen for arbitrary durations before being abruptly interrupted by the next loop. A player getting stuck on a puzzle for multiple hours can become a unique aspect of the player’s progression. Is that aspect of their story best described by an eight measure loop that continues for multiple hours?

The question is even more relevant when considering that the technology-driven video game industry grosses over one-hundred billion dollars a year (Batchelor, 2018). The video game industry employs professional writers, artists, animators, programmers, sound designers, and composers to work in an artistic medium that aims to evoke an emotional response from players by giving them a means to empathize with, and control, a character through a meaningful virtual experience. Why are more resources not given to developing better DMSs?

The primary research question of this thesis is:
How can the design of a DMS allow the temporal qualities of musical structure to communicate a musical narrative that is congruent with a player’s unpredictable gameplay-driven narrative?

The research question may suggest that repetition is what prevents music from being congruent with player action, but that is not completely true. The problem is when music becomes ‘stuck’ or fails to remain congruent with the player’s experience of the narrative. Observations about when the music becomes stuck and whether or not it is congruent with the player’s experiences are, to an extent, subjective. The observations can be made from an aesthetic perspective depending on the style of music, how the music changes, when it changes, and how it relates to the player’s experience. Therefore, the DMS could be designed differently according to these aesthetic criteria to allow for the music to develop more ‘effectively’. This type of subjective critical observation and analysis does not offer empirical data about how ‘effective’ a DMS is. However, it is useful to me as a composer to inform how I approach my own DMS designs.



1.1.3. - Statement of the Problem

The conflict between musical structure and interactivity is articulated by Stevens and Raybould:

“Given that musical training unsurprisingly leads to a heightened sensitivity (Dellacherie et al. 2011), it may be that many commentators with a background in music (such as ourselves) are prone to exaggerate the problems that arise when such patterns of expectancy are interrupted by the need to respond to game events, but there is strong evidence that no formal training is required to make automatic predictions of chord functions (Koelsch 2011), to be acutely aware of phrase boundaries (Nan, Knösche, and Friederici 2009) and expectations of metrical or pitch patterns (Huron 2006), and that breaking these patterns of expectation can cause disorientation (Margulis, 2007) and negative responses (Steinbeis, Koelsch, and Sloboda 2006)” (Stevens and Raybould, 2014, 149; citations as in quote).

Trevor Rawbone, in an article on Melodrive’s website, refers to this as a problem of musical congruence. Rawbone cites that Raybould’s solution to this problem is to compose music that has elements of indeterminacy, as to avoid any sort of musical vector that restricts the possible ‘correct’ developmental pathways (Rawbone, 2018). Rawbone writes:

“…[W]hile the approach of creating indeterminacy and avoiding vectors, as well as having a catalogue of pre-prepared music, may be satisfactory solutions, we…feel it is a shame for music to lose such interesting goal-oriented concepts such as melody, metrical structure, and harmonic structure, because these overarching structures provide a lot of interest in music”, later adding, “I argue that if musical elements weren’t congruent with each other, we wouldn’t be able to understand music at all. That is, if the elements of music were totally noncongruent, no part would be by definition relatable to the next, and so it would be impossible for such music to be meaningful” (Rawbone, 2018).

Rawbone’s quotation is a subjective criticism of Raybould’s proposed solution, which is to avoid the use of musical elements (such as melody, metrical structure, and harmonic structure) that could potentially restrict the flexibility of the DMS. Rawbone argues that these elements are part of what make music interesting and that a solution to music interactivity should not be at the cost of these elements. I tend to agree with Rawbone’s argument that there must be a solution for creating DMSs that are able to maintain these various musical components while also being highly flexible. I attempt to feature these various musical components in my DMS prototype in Chapter 5.



1.1.4. - Primary, Secondary, and Subsidiary Contributions

The DMS prototype found in Chapter 5 is the primary composition contribution of the thesis. The prototype is a playable game that demonstrates a variety of dynamic music techniques used within a complex conditional logic system. One of the novelties of the system is that it is designed to reference previously played musical segments (using predefined conditions) to determine what the most suitable music for the player’s current actions are within the occurring musical structure. The prototype may only take a few minutes to complete, however, there is over thirty minutes of music that can be heard depending on the sequence and timing of different player actions. The prototype contains music of different styles implemented into three sections of gameplay. The first section is a non-linear area where the player must explore and collect keys, the second is a linear area where the player encounters an enemy, and the third contains a scripted gameplay event followed by a boss fight. The prototype’s DMS design is influenced by my conclusions from three analytical case studies presented in Chapter 4.

The secondary contribution of my thesis is the original analytical method and model I have developed for understanding how DMSs are designed to output music as a structure defined by player actions. The goal of the analytical model is to provide the analyst with a technical blueprint of the DMS, which allows them to pin-point specific musical experiences and what gameplay actions cause them. Chapter 4 reviews existing analytical models and methods, and introduces my own.

Chapter 3 provides the reader with a means of learning about the various current techniques that video game composers use to create DMSs. An in-depth understanding of current techniques is required for making sense of my analysis and prototype. In addition to being a prerequisite to later chapters, Chapter 3 is a contribution in that it reviews existing literature about video game composition techniques, consolidates synonymous terminologies used across the industry and academia, shows contemporary examples of how various techniques are used in current games, and demonstrates how different techniques are used in four separate playable mini-games that I have made.

Chapter 2 reviews the history of dynamic video game music development. The purpose of Chapter 2 is to reveal the various ways composers have created dynamic music throughout the history of game development and how that has led to today’s practices. In Chapter 2 I cross reference various historical studies with informal discussions I have had with some of the audio professionals who were actively working in the game industry when the standard practice shifted from using sound generators, to MIDI, and then to rendered audio assets. Understanding why composers are practicing specific methods within the industry is important for understanding the context of my research, given my prototype uses current industry practices as a foundation.

“Music for games plays an essential role in building these worlds of sci-fi fantasy or alien invasion or medieval exploration, through the way that players relate to the characters they control. It’s as if the music is the sound of a spell that magically transfers our agency as players to the avatar on screen, whether it’s Mario, Link or Kirby. In video games we play with music and are played by the music. It’s a powerful digital-musical alchemy, rich with potential. Right, back to that boss in Zelda. I will conquer you, Calamity Ganon! Oh. I wonder how many thousands of times I’ll have to hear this battle music before I finally vanquish him?" -Service, 2018

__________________________________________________

Chapter 1.2. Literature Review Summary

Throughout the thesis I introduce various topics to either contextualize my research or provide prerequisite information required for understanding a more complex concept. This literature review summary has been written to introduce the reader to the various topics and sources used throughout the thesis. The literature is more extensively reviewed throughout each chapter as different topics are shifted into focus.



1.2.1. - Dynamic Music Techniques

The study of current dynamic music techniques used in video games is relatively limited within the academic field of ludomusicology. Information regarding new innovations and industry practices is mostly contributed by professionals who work in the game industry, presented in blogs, textbooks, interviews, and presentations given at developer conferences. Interactive music research has also been conducted in the field of computer-generated music using machine learning and real-time algorithmic processes. Interactive music and adaptive form may also be considered from the perspective of concert music.

1.2.1.1. - Blogs, textbooks, interviews, and presentations by industry professionals
Interviews with industry composers can be found online on a number of blogs, and are primary sources for understanding what game audio professionals are doing in the industry. Guy Whitmore, considered a pioneer of adaptive music techniques, has been interviewed (Velardo, 2014; Brandon, 2012; PopCapChannel, 2014) about his work and expresses the importance of adaptive music in games. He has written articles that discuss methods, technical challenges, and creative processes for creating adaptive music (Whitmore, 2003). Many of the compositional techniques that Whitmore discusses are reviewed in Chapter 3.

Marty O’Donnell, the composer for the Halo franchise, describes his experiences working with different game development teams as an audio director (Audy-Rowland, 2004), and has spoken about more specific DMS design concepts and creative collaborations at GDC (Game Developers Conference) (O’Donnell, 2011; Johnson et al., 2008). O’Donnell is responsible for developing a custom adaptive music engine for “Halo: Combat Evolved” (Bungie, 2001), which was one of the first games to extensively utilize horizontal resequencing techniques with streaming audio. Chapter 2 looks more closely at how Halo:CE was able to demonstrate the capability of adaptive streaming music techniques, and Chapter 4 contains an analysis of Halo:CE’s DMS.

Michael Sweet has written a textbook called “Writing Interactive Music for Video Games” (Sweet, 2015), which explores various audio production techniques, dynamic music composition techniques, historical examples of adaptive musical forms, and video game music business. Other textbooks that I believe are a significant contribution to understanding dynamic music techniques are Chance Thomas’ “Composing Music for Games: The Art, Technology, and Business of Video Game Scoring” (Thomas, 2016), which includes examples and interviews from many of the industry’s leading composers; Winifred Phillips’ “A Composer’s Guide to Game Music” (Phillips, 2014), which includes theoretical topics such as how music helps create immersion in games and what the categorical roles and purposes that music in games can serve; and Stevens & Raybould’s “Game Audio Implementation: A Practical Guide Using The Unreal Engine” (Stevens & Raybould, 2016), which includes technical knowledge about how video game music is written and implemented in the UE4 game engine.

Throughout this literature there is a variety of terminology that has been invented to describe dynamic music techniques. Much of the inventive language used is redundant or synonymous with existing terms, which leads to unnecessary confusion about whether or not the techniques being described are somehow different. As part of Chapter 3, where I explore and demonstrate various dynamic music techniques, I attempt to identify and consolidate terminology into broader categorical functions.

Video game development conferences such as GDC, Develop, GameSoundCon, Unite Copenhagen, and Ludomusicology offer platforms for game audio professionals to give presentations about how they are using dynamic music in their games. GDC, perhaps the biggest professional game development convention, archives all of the presentations online in their ‘GDC Vault'. There is a rich variety of presentations that include: discussion about how different aspects of music composition can contribute to improving adaptive music (Bajakian, 2010), how licensed pop songs can be processed in real-time to adapt to the timing of player actions (Durity, 2013), how generative music systems can be designed to create a more varied adaptive musical experience for the player (Hedges, Larson, Mayer, 2010), the creative process for creating unique interactive scores that mesh electronic sample-based libraries with recorded orchestral stems (Deriviere & Kurlander, 2014), and the previously mentioned GDC talks by Marty O’Donnell (O’Donnell, 2011; Johnson et al., 2008). Developer conferences not only offer direct insight from audio professionals and composers, but they also offer a network of industry professionals who are often excited to share new ideas and projects.

1.2.1.2. - Machine learning and real-time algorithmic processes
The development of adaptive music is not exclusive to video games. Many of the techniques and approaches to dynamic music within the industry are a result of how games are made and how the history of games have evolved. Within the field of computer music, techniques have been developed that could offer solutions to problems that game composers face, such as real-time music generation based on continuously changing parameters. Chapter 2 describes how real-time generative audio techniques were the standard for early game music. However, real-time algorithmically generated composition is not a common practice.

The earliest computer-generated composition was Hiller and Issacson’s "Illiac Suite" (Hiller & Issacson, 1957) (Maurer IV, 1999), which predates one of the earliest video games, “Pong” (Atari, 1972), by fifteen years. The prospect of using a computer to generate music offline (not in real-time) has always been an option for easing the workload of composition work. Companies like Melodrive and AIVA use AI systems to generate musical material that can later be produced and rendered. But most game developers tend to hire a composer, or even a team of composers, as was done for the soundtrack of “Destiny 2” (Bungie, 2017).

Real-time algorithmically generated music is scarcely used in the industry, however there are some examples. “C.P.U. Bach” (1993), created by Sid Meier and Jeff Briggs for the 3DO Company (The Strong National Museum of Play, 1994), used algorithms to generate music in the style of J.S. Bach. Another example of real-time algorithmic composition in games is Brian Eno’s soundtrack to “Spore” (Maxis, 2008). In Spore, various parameters are used to generate music that evolves alongside the organisms’ evolution in the game. Baldur Baldursson, composer for “EVE Online” (CCP, 2003), has described how Kjartan Olafsson and himself have been creating a generative music system called CALMUS, which outputs the music as MIDI data in real-time and feeds the MIDI data into Wwise where virtual instruments are hosted. Wwise generates music to play alongside gameplay (Phillips, 2016). The main problem with real-time computer-generated music is that when sequencing virtual instruments in real-time, the quality of the instrument samples must be restricted so that it does not drain the computer’s computational resources (Sweet, 2015, 206). Real-time computer-generated music is more common when the aesthetic of low-fidelity synthesized instruments is more acceptable, as is in “Peggle” (PopCap Games, 2007).

For the most part, algorithmically-generated music has not been valued as a method for creating dynamic music in games. However, as the computation power of personal computers increase, cloud computing/gaming becomes more popular (AWS, Orion, Microsoft Azure, Stadia), machine-learning becomes more fluent in music composition aesthetics, and audio generation/synthesis becomes more realistic (NSynth, Magenta Studios, GANSynth, Music Transformer, WaveNet), it becomes more probable that real-time computer generated music may become a standardized paradigm.

1.2.1.3. - Concert music
Adaptive music in the concert setting, or music which has interchangeable or movable forms, dates back as early as Mozart’s Musikalisches Würfelspiel (Maurer IV, 1999). Throughout the twentieth and twenty-first centuries composers experimented more freely with open or indeterminate musical structures. The serialization of various musical parameters in pieces such as “Le Marteau Sans Maître” (Boulez, 1954), “Kinderstück” (Webern, 1924), “Mode de valeurs et d’intensites” (Messiaen, 1950), “Philomel” (Babbitt, 1964), and others, may be considered a precursor to the use of mathematical parameters in games to dictate musical change. The indeterminacy of musical development in games can be thought to have a relationship with works such as “Music of Changes” (Cage, 1951), “Mosaic Quartet” (Cowell, 1935), “Projection 2” (Feldman, 1951), or “The Metaphysics of Notation” (Applebaum, 2008).

Current video game practices use parameters of gameplay, which are determined in real-time by the player. These parameters dictate the structure of the music. Contemporary music compositions that use open, modular, or mobile forms, such as “Klavierstück XI” (Stockhausen,1956), “Cobra” (Zorn, 1984), and “In C” (Riley, 1964), share this concept, in which different sections of music are written to be interchangeable with each other depending on indeterminate factors.

The composition and theory of open form and adaptive music is not unique to games. However, the production of games is often commercially motivated and presents challenges that experimental music throughout the twentieth and twenty-first century are not necessarily required to confront. In particular, much of the experimental concert music that may provide insight into new ways of approaching adaptive game music, does not offer solutions that are applicable to music that must use a more traditionally tonal syntax. This will be discussed in Chapter 1.3. Because of this, experimental compositional research for the concert setting does not directly inform the development of my own methods. However, my long preexisting familiarity, fondness, and experiences with writing serial, open form, and indeterminate concert music is inherently ingrained into how I think about dynamic music.



1.2.2. - Understanding Video Game Music

Analyzing DMS design from a compositional and technical perspective seems to be a less common area of study compared to other research topics within the field of ludomusicology. Even though some composers share their creative solutions to dynamic game music in blogs, interviews, and presentations, there are few methods developed for analyzing and interpreting how DMSs function from the gamer’s perspective.

1.2.2.1. - Analyzing and interpreting DMS function from the gamer’s perspective
Summers’ chapter about methods from his book “Understanding Video Game Music” offers the most conclusive foundation for approaching video game music analysis (Summers, 2016, 36-50). Summers discusses different sources from which the analyst may uncover information for understanding how video game music functions. Summers introduces how an analyst must engage with games as a “critical player”, interacting with the game as both a player and researcher (Summers, 2016, 34). Chapter 3 looks more closely at Summers’ analytical method and how it relates to my own.

Whalen introduces a graph of what he describes as Syntagmatic v. Paradigmatic relationships in “Super Mario Brothers” (Nintendo, 1985) to describe the structure of musical sequences (Whalen, 2004). Brame creates a representation of game-score plot-connections between different parts of games that have different musical segments associated with them. His model describes how various musics associated with different parts of gameplay can interact with each other (Brame, 2009). While both Brame’s and Whalen’s models provide information about how DMSs function, neither show how DMSs are constructed in enough detail for my research goals.

Medina-Gray has developed a model that illustrates individual loops, layers, and segments, and how they could potentially be assembled to create a variety of musical outcomes. Medina-Gray’s model requires the reader to reference a toolkit of symbols devised to describe different aspects of the DMS such as musical smoothness (Medina-Gray, 2014). Karen Collins creates an illustration of how the DMS in the game Russian Squares develops over the span of a single gameplay session, notating a score that contains a numbered list of short musical phrases and then layering different blocks together; each block is labeled with a number from the list (Collins, 2008, 154). Tim Summers’ analytical model resembles a storyboard containing screen shots from the game (Summers, 2016, 25). The storyboard contains three sections. The first section is for describing the trigger that causes music to change. The second section is for showing the musical structure. And the third section is for describing the music from each part of the gameplay. Each of these analytical methods seek to illustrate specific aspects of how DMSs function in games, and has informed the development of my own analytical method and model. None of the models, however, were ideal for my research goals. This claim, along with the aforementioned literature, is discussed in greater depth in Chapter 4, where I engage with it more critically.

1.2.2.2. - Research topics within the field of ludomusicology
Understanding the broader range of topics within the field of ludomusicology helps to more generally contextualize research about analyzing and designing DMSs. Isabella van Elferen describes how the convergence of musical literacy, affect, and interaction can effect the player’s sense of immersion during gameplay (Van Elferen, 2016). While all three categories are essential contributors to immersion, my thesis is centered on the topic of interaction.

Melanie Fritsch explores how the participatory culture of video games can develop a literacy of what games are, and their broader cultural meaning to those who are involved (Fritsch, 2016). Through this, video games become objects with cultural meaning outside of the game itself, where gamers, to some extent, develop the meaning of the game beyond its initial creation. Fritsch cites the research of Zagal (2010), outlining the part of game literacy that requires players to have “the ability to understanding meanings with respect to games” (Zagal, 2010, 23; from Fritsch, 2016). Within the scope of that understanding, Zagal continues by describing that part of what that means is to be able to deconstruct and understand various game components, “and how they facilitate certain experiences in players” (Zagal, 2010, 23). My goal of analyzing how DMSs function on a technical level is very much in line with wanting to understand how specific components of the DMS facilitate experiences within players.

The “Oxford Handbook of Interactive Audio” (Collins, Kapralos, and Tessier; 2014), offers a range of ludomusicological topics spanning from cognitive psychology and perception to how methodological and technological developments could impact game audio processes in the future. Within the handbook, however, there is nothing that directly addresses the topic of analyzing or creating DMSs from a technical perspective.

Tim Summers has compiled an outline titled “How to Hear a Video Game: An Outline” (Summers, 2016, 208-214). The outline is populated with a number of examples that Summers identifies as being notable areas for research. Topics include different styles and genres of music, different ways music can function in games, the conceptual implications of diegetic vs. non-diegetic music, the composer’s relationship with the game making process, cultural and political implications of how game music is used in and outside of games, and more. Many of the topics have been researched by the ludomusicology community, and the outline would be presumably expanded with a growing number of new topics and references as the field continues to develop.

__________________________________________________

Although the field of ludomusicology is still relatively young, there is an extensive depth to the variety of research seeking to understand the significance of how game music has woven its way into society. My research is purely a practical approach to understanding what composers are currently doing in the game industry, how they are doing it, and how I can build on their methods to solve problems that exist between player-interactivity and musical structural development. The most valuable resources for my research has been the literature contributed by industry professionals who openly share their methods and ideas, and being able to play games that have DMSs. Academic ludomusicology literature tends to be theoretically focused, exploring topics that are tangential to my own.

Lastly, a contributing factor to my research output has been formed by my experience working in the industry with game developers and publishers to produce sound design and dynamic music for games. This has provided me with the experience of collaborating with interdisciplinary artists, confronting the challenges I propose throughout this thesis in a professional context, and discussing game audio within a professional network of composers, sound designers, and audio directors from AAA and indie development studios.

__________________________________________________

Chapter 1.3. Musical Style

A major part of the criticism presented throughout this thesis relates to the music’s structural development alongside gameplay. Different video games require different styles of music, not all games are narrative-driven, not all narrative games are non-linear, and not all non-linear games require a style of music that is relevant to my criticisms about musical structure. Therefore it is important to establish what styles of music are being used and why.

My research interests pertain to non-linear and semi-linear story-driven games that allow the player, in some way, to dictate the sequence of narrative events at their own pace. Within most story-driven games music does not span the entire duration of gameplay. Music is often used to emphasize different moments of the gameplay, with ambience or silence laced between those moments when there is less drama. Depending on the style of music being used, different structural and developmental rules may be expected. When critically assessing existing DMSs in games it is necessary to identify the way a player interacts with music through the gameplay and what the music is trying to achieve.

The player’s interaction with music can be defined as the passing of information between two systems that results in changes to musical detail and form (Graugaard, 2005, 149). Perhaps the most substantial amount of adaptive musical research has been pursued in the context of aleatoric, open, or modular forms — a “score subject to physical manipulation of its components, resulting in an unknown number of different, integral, and ‘valid’ realizations” (Brown, 1954). While the interactive and adaptive components of musical scores by composers such as Cage, Brown, Zorn, Stockhausen, and others seem relevant to the goals of video game DMSs, the theory that allows most of these examples to function has not necessitated an adherence to traditional harmonic, melodic, or structural rules (Sweet, 2015, 110-120). This is an important point as the vast majority of video game music often operates within a traditionally tonal syntax. This argument further illustrates why it is essential to contextualize the discussion about ‘adaptive music’ within a specific musical style, as the techniques that allow the structures of (for example) Earle Brown’s music to function dynamically would not be completely relevant when trying to develop a DMS which allows the structure of Beethoven’s music to function dynamically.

Perhaps the most relevant historical adaptive musical piece would be Mozart’s Musikalisches Würfelspiel, which has many parallels with how contemporary DMSs are designed to facilitate a type of traditionally tonal music. Mozart’s dice game was played by rolling dice in order to “assemble a number of small musical fragments, and combining them by chance, piecing together a new piece from randomly chosen parts” (Alpern, 1995, from Maurer IV, 1999). Contemporary DMSs break music into small musical fragments that are later reassembled in a sequence that depends on player-dictated parameters, much like Musikalisches Würfelspiel. While Mozart’s game is a great historical example, the development of adaptive music theory within the context of music that relies on tonal harmonic function seems to mostly exist within the development of music for games. A history of the development of dynamic music in video games is reviewed in Chapter 2.2.

In addition to specific genres of games and art styles informing the style of music a composer must write, creative limitations also arrive at the advent of commercial interest. Video game publishers generally oversee the entire development process of the game by providing financial backing and marketing strategies. Because of this, publishers are most concerned with the creation of a product that is marketable to sell worldwide (Collins, 2008, 85). The creative output of a game composer is partly a collaboration with publishers that impose restrictions based on what they believe is most profitable.

Whalen states, “In cinema, the equivalent approach might hold that studying film music must begin with understanding the audience as opposed to situating music within the semiotic apparatus of the film itself…” (Whalen, 2007, 68). Whalen’s statement about the parallels between film and video game music describes a situation in which the music must be observable as something that serves the consumer’s expectation. This further suggests the importance of a DMS that is capable of adhering more closely to the structural development implied by specific musical styles. Expectations of musical style hinge on a number of technical and cultural motivations. Summers describes this as a type of intertextuality between different media industries and gaming culture (Summers, 2016, 40).

For example, a survival horror game may appropriate an atonal and textural score. This may be an easier harmonic context to create a seamless DMS with. The composer may have more freedom to create contrasting branching musical segments by changing density, orchestration, texture, and harmonic tensions. However, if the video game takes place in a Victorian English setting that appropriates a nineteenth century inspired score, then the composer would have the more difficult task of designing a DMS in which the music’s multiple branching possibilities adheres to strict nineteenth century compositional rules. For the purpose of this thesis I will be analyzing and composing a wide selection of musical styles.

__________________________________________________


Previous: Acknowledgements
Next: Chapter 2: History and Development of Game Music
Contents: Back to Table of Contents